EDA
We can first visualize the data after dimension reduction with zinbWave. We then use t-SNE and color the cells with the allen labels. Since we use various values for the k parameter, we plot two representations with the most extreme values of k.

Reduction in the number of clusters

Improvement in ARI


LS0tCmF1dGhvcjogIkhlY3RvciBSb3V4IGRlIELDqXppZXV4IgpkYXRlOiAnYHIgZm9ybWF0KFN5cy50aW1lKCksICIlZCAlQiAsICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2RlcHRoOiAyCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIGNvZGVfZG93bmxvYWQ6IFRSVUUKICAgIApwYXJhbXM6CiAgZGF0YXNldDogIjEweF92M19jZWxsc19NT3AiCiAgdGl0bGU6ICJBbmFseXNpcyBvZiB0aGUgMTB4X3YzX2NlbGxzX01PcCBkYXRhc2V0IgotLS0KLS0tCnRpdGxlOiBgciBwYXJhbXMkdGl0bGVgCi0tLQoKYGBge3IgbG9hZCBwYWNrYWdlcywgaW5jbHVkZT1GfQpsaWJyYXJ5KGtuaXRyKQpvcHRzX2NodW5rJHNldCgKICBmaWcucG9zID0gIiFoIiwgb3V0LmV4dHJhID0gIiIsIHdhcm5pbmcgPSBGLCBtZXNzYWdlID0gRiwKICBmaWcud2lkdGggPSA1LCBmaWcuYWxpZ24gPSAiY2VudGVyIiwgZWNobyA9IEYKKQpsaWJzIDwtIGMoImhlcmUiLCAiZHBseXIiLCAiZ2dwbG90MiIsICJ0aWR5ciIsICJzdHJpbmdyIiwgInJlYWRyIiwgImNvd3Bsb3QiLAogICAgICAgICAgImNsdXN0ZXJFeHBlcmltZW50IiwgIm1jbHVzdCIsICJSQ29sb3JCcmV3ZXIiLCAicHJvZ3Jlc3MiLCAiRHVuZSIsCiAgICAgICAgICAicG5nIiwgInB1cnJyIikKc3VwcHJlc3NNZXNzYWdlcygKICBzdXBwcmVzc1dhcm5pbmdzKHNhcHBseShsaWJzLCByZXF1aXJlLCBjaGFyYWN0ZXIub25seSA9IFRSVUUpKQopCnJtKGxpYnMpCnR5cGUgPC0gZnVuY3Rpb24oZGF0YXNldCkgewogIGlmIChzdHJfZGV0ZWN0KGRhdGFzZXQsICJTTUFSVCIpKSByZXR1cm4oIlNtYXJ0LVNlcSIpCiAgaWYgKHN0cl9kZXRlY3QoZGF0YXNldCwgIjEweF92MyIpKSByZXR1cm4oIjEwWF9WMyIpCiAgaWYgKHN0cl9kZXRlY3QoZGF0YXNldCwgIjEweCIpKSByZXR1cm4oIjEwWCIpCiAgaWYgKHN0cl9kZXRlY3QoZGF0YXNldCwgIlJlZ2V2IikpIHJldHVybigiUmVnZXYiKQp9CnR5cGUgPC0gdHlwZShwYXJhbXMkZGF0YXNldCkKbWVyZ2VycyA8LSByZWFkUkRTKGhlcmUoImRhdGEiLCAiRHVuZSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhc3RlMChwYXJhbXMkZGF0YXNldCwgIl9tZXJnZXJzLnJkcyIpKSkKYGBgCgojIEVEQQoKV2UgY2FuIGZpcnN0IHZpc3VhbGl6ZSB0aGUgZGF0YSBhZnRlciBkaW1lbnNpb24gcmVkdWN0aW9uIHdpdGggemluYldhdmUuIFdlIHRoZW4gdXNlIHQtU05FIGFuZCBjb2xvciB0aGUgY2VsbHMgd2l0aCB0aGUgYWxsZW4gbGFiZWxzLiBTaW5jZSB3ZSB1c2UgdmFyaW91cyB2YWx1ZXMgZm9yIHRoZSBrIHBhcmFtZXRlciwgd2UgcGxvdCB0d28gcmVwcmVzZW50YXRpb25zIHdpdGggdGhlIG1vc3QgZXh0cmVtZSB2YWx1ZXMgb2Ygay4KCmBgYHtyIHQtU05FIHBsb3RzLCBmaWcud2lkdGg9MTgsIGZpZy5oZWlnaHQ9OX0KcGxvdHMgPC0gbGlzdC5maWxlcyhoZXJlKCJGaWd1cmVzIiwgIkVEQSIpKQpwbG90cyA8LSBwbG90c1tzdHJfZGV0ZWN0KHBsb3RzLCBwYXJhbXMkZGF0YXNldCldCnBsb3RzIDwtIHBsb3RzW3N0cl9kZXRlY3QocGxvdHMsICJwbmciKV0KcGxvdC5uZXcoKQpwbG90LndpbmRvdygwOjEsIDA6MSkKcmFzdGVySW1hZ2UocmVhZFBORyhoZXJlKCJGaWd1cmVzIiwgIkVEQSIsIHBsb3RzWzFdKSksIDAsIDAsIC41LCAxKQpyYXN0ZXJJbWFnZShyZWFkUE5HKGhlcmUoIkZpZ3VyZXMiLCAiRURBIiwgcGxvdHNbbGVuZ3RoKHBsb3RzKV0pKSwgMC41LCAwLCAxLCAxKQpgYGAKCgojIEdlbmVyYWwgY29tbWVudHMKCldlIGhhdmUgdGhlIGZvbGxvd2luZyB3b3JrZmxvdwoKYGBge3Igd29ya2Zsb3csIGZpZy5oZWlnaHQ9NiwgZmlnLndpZHRoPTEwfQpwbG90Lm5ldygpCnBsb3Qud2luZG93KDA6MSwgMDoxKQpyYXN0ZXJJbWFnZShyZWFkUE5HKGhlcmUoIkV4cGxhaW5hdGlvbnMiLCAiV29ya2Zsb3cucG5nIikpLCAwLCAwLCAxLCAxKQpgYGAKCkluIHRoZSBTZXVyYXQgY2x1c3RlcmluZywgdGhlcmUgYXJlIHR3byBwYXJhbWV0ZXJzIHRvIGNob29zZSBmcm9tOiB0aGUgcmVzb2x1dGlvbiBhbmQgdGhlIGsucGFyYW0gKHVzZSBpbiBhIGtubiBzdGVwKS4gRnJvbSB0aGUgc2V1cmF0IGhlbHAgZmlsZSwgZm9yIHRoZSByZXNvbHV0aW9uIHBhcmFtZXRlcjogdXNlIGEgdmFsdWUgYWJvdmUgKGJlbG93KSAxLjAgaWYgeW91IHdhbnQgdG8gb2J0YWluIGEgbGFyZ2VyIChzbWFsbGVyKSBudW1iZXIgb2YgY29tbXVuaXRpZXMuCgpXZSBwaWNrZWQgYSB2YWx1ZSBvZiAxLjYgYXMgd2Ugd2FudCBtYW55IGNsdXN0ZXJzIHRvIHN0YXJ0IHdpdGguIFdlIGFsc28gcGljayBrID0gNTAuIFRob3NlIHZhbHVlcyBzZWVtIGludGVybWVkaWF0ZSBpbiB0ZXJtIG9mIEFSSSB3aXRoIG90aGVyIHBhaXJzIG9mIHBhcmFtdGVycyAoc2VlIHNlY3Rpb24gU2V1cmF0IEFSSSBwYXJhbSkKCldlIGFsc28gY29uc2lkZXIgdHdvIHdheXMgb2YgY29tcHV0aW5nIHRoZSBBUkkgYmV0d2VlbiBSU0VDIGFuZCBvdGhlciBtZXRob2RzLiBFaXRoZXIsIHdlIGZvcmNlIFJTRUMgdG8gYXNzaWduIGFsbCBjZWxscyB0byBhIGdpdmVuIGNsdXN0ZXIsIG9yIHdlIG9ubHkgY29tcHV0ZSB0aGUgQVJJIGJldHdlZW4gUlNFQyBhbmQgdGhlIG90aGVyIG1ldGhvZHMgb24gdGhvc2UgY2VsbHMgdGhhdCBSU0VDIGRvIGNsdXN0ZXIuIE5vdGUgdGhhdCBpbiB0aGUgc2Vjb25kIGNhc2UsIG90aGVyIHBhaXJzIG9mIG1ldGhvZHMgYXJlIGNvbXBhcmVkIHVzaW5nIGFsbCBjZWxscy4gV2UgZGVub3RlIGFzIFJzZWNUIHRoZSBjbHVzdGVyIGFzc2lnbmVtZW50IHdoZXJlIGFsbCBjZWxscyBhcmUgYXNzaWduZWQgKGkuZSBSc2VjIFRvdGFsKS4gCgpUaGUgQVJJIG1lcmdpbmcgbWV0aG9kIHdvcmtzIGFzIGZvbGxvdy4gV2UgaXRlcmF0ZSBvdmVyIGFsbCBwYWlycyBvZiBjbHVzdGVycyBmb3IgZXZlcnkgY2x1c3RlcmluZyBtZXRob2QuIEZvciBlYWNoIHBhaXIsIHdlIHRyeSB0byBtZXJnZSB0aGUgdHdvIGNsdXN0ZXJzIGFuZCBzZWUgaG93IGl0IGltcHJvdmVzIHRoZSBBUkkgd2l0aCB0aGUgb3RoZXIgbWV0aG9kcy4gV2UgdGhlbiBtZXJnZSB0aGUgcGFpciB0aGF0IGltcHJvdmVzIHRoZSBBUkkgdGhlIG1vc3QuIFdlIHN0b3Agd2hlbiB0aGUgQVJJIGNhbm5vdCBiZSBpbXByb3ZlZCBhbnltb3JlLgoKSW4gdGhlIGdlbmVyYWwgY2FzZSwgd2UgcGVyZm9ybSB0aGUgQVJJIG1lcmdpbmcgd2l0aG91dCB0aGUgYWxsZW4gbGFiZWxzIGFuZCB3ZSB0aGVuIHVzZSBpdCBhcyBhIGNvbXBhcmlzb24uIEEgcXVpY2sgb3ZlcnZpZXcgb2YgaG93IHRoZSBhbGdvcml0aG0gcGVyZm9ybXMgd2l0aCB0aGUgYWxsZW4gbGFiZWxzIGlzIHNlZW4gYXQgdGhlIGVuZC4KCiMgUmVkdWN0aW9uIGluIHRoZSBudW1iZXIgb2YgY2x1c3RlcnMKCmBgYHtyIEltcCBubyBhbGxlbn0KcGxvdFByZVBvc3QobWVyZ2VycykKYGBgCgojIEltcHJvdmVtZW50IGluIEFSSQoKYGBge3IgQVJJIGltcCBubyBhbGxlbiBjZWxsfQpwbG90X2dyaWQoCiAgcGxvdEFSSXMobWVyZ2VycyRpbml0aWFsTWF0KSArIGdndGl0bGUoIkJlZm9yZSBtZXJnaW5nIiksCiAgcGxvdEFSSXMobWVyZ2VycyRjdXJyZW50TWF0KSArIGdndGl0bGUoIkFmdGVyIG1lcmdpbmciKQopCmBgYAoKCmBgYHtyIEFSSSB0cmVuZCwgZmlnLndpZHRoPTl9CkFSSXRyZW5kKG1lcmdlciA9IG1lcmdlcnMpCmBgYA==